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DETAILED ACTION 
Response to Amendment 

The amendment filed 05/14/2007 has been entered and made of record. The 
remarks concerning the 35 USC 101 rejection made in the Non-Final office action filed 
02/13/2007 has been considered. The 35 USC 101 rejection is withdrawn. Claims 30, 
31, and 33-36 have been cancelled. Claims 1-29, 32, and 37 are pending. 

Please note: Attempt to reply to Interview Request filed 05/14/2007 was made, but no 
response was given. 

Response to Arguments 

In regards to claim 1, Applicant's arguments, filed 12/19/2006, have been fully 
considered and are persuasive. Therefore, the rejection has been withdrawn. 
However, upon further consideration, a new ground(s) of rejection in view of Ohshima et 
al. (2003/0032484 A1) in view of Richey (5,310,794), Lyons (6,181,343 B1) and Sauer 
(2002/0140708 A1) is made below. Applicant argues the newly amended claims require 
the integrated scene is directly viewable by the individual. The new grounds of rejection 
made in view of Ohshima et al. (2003/0032484 A1) in view of Richey (5,310,794), Lyons 
(6,181,343 B1) and Sauer (2002/0140708 A1) teaches being able to view virtual images 
without having to use an HMD [0052]. 
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Claim Rejections - 35 USC § 103 

The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

1. Claim 32 is rejected under 35 U.S.C. 103(a) as being unpatentable over Richey 

(5,310,794) in view of Lyons (6,181,343 B1). 

Richey teaches the limitations of claim 32 with the exception of disclosing 
updating the sequence of images. However, Lyons teaches manipulating the 
information on a large-screen display, in response to interaction with the user. 

Richey discloses a panoramic image based virtual reality display system. 
Referring to Fig. 34, the viewer's entire body is positioned in the large display assembly 
(23), in which display units surround the viewer such that the viewer sees a respective 
portion of the scene of spherical coverage in any viewable direction. The large display 
assembly (23) is comprised of a structural framework (9) and supports (10), which hold 
the display units (11) and optical enlarging means (12) securely in place [col. 9 lines 14- 
23)]. The floor (130) and its associated display unit (11) beneath, to the sides, and over 
the viewer/operator are integrated so the viewer is presented with a substantially 
continuous scene for viewing [col. 28 lines 15-19]. Display systems and optical 
enlargement means mounted on spring-hinged doors, latches, or rollers, allow the entry 
and exit assembly (131) to move back and forth in an open and closed position to 
enable viewer entry and exit [col. 28, lines 28-32]. These means may also be used for 
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easy assembly and disassembly. Richey discloses that it is further foreseen that the 
optical and camera arrangements disclosed in Figs. 6-17 may transmit their recorded 
image to various types of sensors such as visual, motion detection, and pyroelectric 
sensors [col. 34 lines 57-61]. 

Lyons teaches a system and method for permitting three-dimensional navigation 
through a virtual reality environment. The video image display means displays three- 
dimensional graphical objects within the virtual reality environment, and movement by 
the system users causes apparent movement of the three-dimensional objects 
displayed on the video image display means so that the system user appears to move 
throughout the virtual reality environment [col. 6 lines 22-28]. Additionally, as described 
in Fig. 6, by taking a step forward, user (62) causes SBIP (60) to move graphical 
images (92) on display screen (54) so that the user (62) appears to be moving forward 
through virtual reality environment (94). Furthermore, the user can move left, right, stop 
or in reverse, and the SBIP (60) updates the graphical images (92) accordingly on the 
display screen (54) [col. 9 line 37 - col. 10 line 10]. 

Therefore, it would have been obvious to one of ordinary skill in the art at the 
time the invention was made to modify the display units of Richey with the SBIP of 
Lyons in order for the user to be in an immersive display. This provides the advantage 
of giving the user the impression of being "in" the virtual reality environment [Lyons: col. 
1 lines 30-41] as well as allowing the users to navigate through and control the objects 
in the virtual reality environment through normal body movements [Lyons: col. 3 lines 
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50-55]. Furthermore, the updating of Lyons adds realism to the interaction with the 
computer [Lyons: col. 6 lines 50-53]. 

2. Claims 1-4, 11-17, 20-23, 29, and 37 are rejected under 35 U.S.C. 103(a) as 
being unpatentable over Ohshima et al. (2003/0032484 A1) in view of Richey 
(5,310,794), Lyons (6,181,343 B1) and Sauer (2002/0140708 A1). 

Ohshima teaches the limitations of claims 1-4, 11-17, 20-23, and 37 with the 
exception of teaching updating the sequence of images and a structure that is large 
enough to accommodate an individual. However, Lyons teaches manipulating the 
information on a large-screen display, in response to interaction with the user and 
Richey teaches a panoramic image based virtual reality display system where the 
viewer's entire body is positioned in the large display assembly (23), in which display 
units surround the viewer such that the viewer sees a respective portion of the scene of 
spherical coverage in any viewable direction. 

In regards to claims 1-4, 14-16, and 37, Ohshima disclose a game apparatus within 
mixed reality. The locations of the player and real object are detected, and the relative 
positional relationship between the player or real object, and the virtual object is 
recognized [abstract]. As shown in Figs. 2 and 3, table (100) has real objects that 
appear in the game. The real objects (101, 104) have location sensors (103) so that the 
virtual images of the game can behave accordingly. The HMD worn by the user 
displays the virtual object (102) as well as the table [0100-0103]. 



Application/Control Number: 10/647,932 Page 6 

Art Unit: 2628 

Sauer teaches an augmented reality environment comprising a stereo display (224) 
of Fig. 1, a 2D/3D graphics module (270), and a video and graphics overlay module 
(222). The two video cameras (210a-b) and the stereo display (224) are employed to 
obtain 3D perception [0052], The two video cameras (210a-b) provide image related 
data corresponding to the real world to the system (200). The data provided by the 
tracking module (212) allows for the relationship(s) between the preceding coordinate 
systems to be known or finally calculated in the pose calculation module (218) [0053]. 

Richey discloses a panoramic image based virtual reality display system. Referring 
to Fig. 34, the viewer's entire body is positioned in the large display assembly (23) 
(claims 1, 37), in which the viewer is surrounded by display units such that the viewer 
sees a respective portion of the scene of spherical coverage in any viewable direction 
(claims 1, 2, 37) [col. 9, lines 14 - 19)]. The assembly is designed to facilitate a single 
or plural number of viewers (claim 16) [col. 28, lines 14 - 15]. The large display 
assembly (23) is comprised of a structural framework (9) and supports (10), which hold 
the display units (11) and optical enlarging means (12) securely in place [col. 9, lines 20 
- 23)]. The floor (130) and its associated display unit (11) beneath, to the sides (claim 
4), and over the viewer/operator (claim 3) are integrated so the viewer is presented with 
a substantially continuous scene for viewing [col. 28, lines 15-19]. Display systems 
and optical enlargement means mounted on spring-hinged doors, latches, or rollers 
(claim 15), allow the entry and exit assembly (131) to move back and forth in an open 
and closed position to enable viewer entry and exit [col. 28, lines 28 - 32]. These 
means may also be used for easy assembly and disassembly (claim 14). Components 
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of the display assembly cooperate to display a substantially continuous panoramic 
scene of spherical coverage about the viewer [col. 9, lines 33]. The panoramic scene 
consists of a plurality of image segments that form a composite image on a single video 
frame (claims 1, 37). Image segments represent portions of camera recorded scene or 
computer graphic information. Typically the segments represent adjacent portions of 
the surrounding panoramic scene. Each image segment is displayed at a designated 
area within the display assembly such that the recorded scene is re-formed in the same 
geometric or geographic orientation in which the scene was recorded [col. 9, lines 48 - 
54]. 

Therefore, it would have been obvious to one in the art at the time the invention 
was made to modify the invention of Ohshima to project the virtual image of the game 
on the wall displays of Richey in order eliminate the use of the HMD implementing the 
stereo displays of Sauer. This would provide realism to the user by not having to wear 
extra equipment to view and interact with the virtual images and real objects of Ohshima 
as if they are same type of object. 

Lyons teaches a system and method for permitting three-dimensional navigation 
through a virtual reality environment. The video image display means displays three- 
dimensional graphical objects within the virtual reality environment, and movement by 
the system users causes apparent movement of the three-dimensional objects 
displayed on the video image display means so that the system user appears to move 
throughout the virtual reality environment [col. 6, lines 22-28]. Additionally, as described 
in Fig. 6, by taking a step forward, user (62) causes SBIP (60) to move graphical 
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images (92) on display screen (54) so that the user (62) appears to be moving forward 
through virtual reality environment (94). Furthermore, the user can move left, right, stop 
or in reverse, and the SBIP (60) updates the graphical images (92) accordingly on the 
display screen (54) [col. 9, line 37 - col. 10, line 10]. 

Therefore, it would have been obvious to one of ordinary skill in the art at the time 
the invention was made to modify the display units of Richey with the SBIP of Lyons in 
order for the user to be in an immersive display. This provides the advantage of giving 
the user the impression of being "in" the virtual reality environment [col. 1 lines 30-41] 
as well as allowing the users the computer users to navigate through and control the 
objects in the virtual reality environment through normal body movements [col. 3 lines 
50-55]. Furthermore, the updating of Lyons adds realism to the interaction with the 
computer [col. 6 lines 50-53]. 

In reference to claims 11-13, Richey illustrates in Fig. 6 the input means for recording a 
panoramic scene of spherical coverage, which is the panoramic camera system 
including a camera (43), and which comprises a portable panoramic video viewing and 
recording system (27), referred to as a panoramic camcorder. The panoramic 
camcorder (27) is carried by a host or vehicle [col. 10, lines 6-13]. It may be obvious to 
one in the art that the images captured by the panoramic camcorder (27) may comprise 
images from the real environment (claim 12). Referring to Figs. 9-12, the optical 
elements (41) are interfaced with the camera (43) to facilitate the composite image (26) 
being transmitted to the recording surface (42) of the camera (43) by conventional 
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means. The recording surface (42) is directly associated with an image processor 
means (44) of a self-scanning solid state imaging device such as a charge coupled 
device located in the image plane of each respective lens element (41) (claim 13) [col. 
11, lines 57-65]. Referring to Figs. 6-10 and 15-17, the electrical section (45) is 
structured to convert the visual images received by the image processor (44) into 
electrical video signals [col. 12, lines 66-68] such that the information is in a format that 
is compatible with standard video processing equipment [col. 13, lines 1-2]. As shown 
in Fig. 6 and 9, the picture signal from the camera (43) is then transferred through 
conductor (46) to a conventional portably structured videotape recorder/player (47) [col. 
13, lines 14-17]. The television signal is then stored by the recorder/player (47) on 
videotape (claim 11) [col. 13, lines 29-30]. Additionally, Lyons teaches displaying live 
video of a camera input of a remote site (claim 13) [col. 7, lines 10-21]. The same 
rationale for combining as applied to claim 1 is incorporated herein. 

Regarding claim 17, it would have been obvious to one in the art that the real objects of 
Ohshima may include objects that coincide with what is displayed on the screen, 
allowing for a simulated environment to the user within the display assembly of Richey 
in view of Lyons: The same rationale for combining as applied to claim 1 is 
incorporated herein. 

In reference to claim 20, Ritchey discloses that is may be foreseen that the display 
assembly [Fig. 32 (23)] may be used as a simulator for various kinds of vehicles [Fig. 55 
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(149)]. Referring to Figs. 54 and 55, the device might take the form of simulated 
controls (150) for such vehicles (149) as a land, sea or air vessel as claimed [col. 34, 
lines 34-40]. The same rationale for combining as applied to claim 1 is incorporated 
herein. 

In regards to claims 21-23, Lyons discloses incorporating audio features to provide 
voice-recognized commands from the system user and sound effects to the display 
screen [col. 13, lines 38-42]. Although Lyons does not explicitly disclose a sound 
generating apparatus, it is implicit that the audio/visual monitor system of Lyons [col. 7, 
lines 65-68] includes some type of sound generating apparatus in order to provide 
sound effects to the display screen. The same rationale for combining as applied to 
claim 1 is incorporated herein. 

In regards to claim 29, Richey discloses a stereographic field of view is arrived by 
sampling left and right eye fields of either of side of the orientation defined by the 
position sensors [Fig. 26 (97)]. To achieve a stereographic effect, image segments [Fig. 
29 (13)] for the left eye and right eye are chosen from two adjacent objective lenses 
[Fig. 17 (37)], each with a different, but adjacent, overlapping field of view of the visual 
scene. Fig. 30 illustrates the resultant stereoscopic image (101) that the image 
processing system has processed for stereographic display [col. 21 lines 50-62]. 

Sauer teaches an augmented reality environment comprising a stereo display 
(224) of Fig. 1, a 2D/3D graphics module (270), and a video and graphics overlay 
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module (222). The two video cameras (210a-b) and the stereo display (224) are 
employed to obtain 3D perception [0052]. The two video cameras (210a-b) provide 
image related data corresponding to the real world to the system (200). The data 
provided by the tracking module (212) allows for the relationship(s) between the 
preceding coordinate systems to be known or finally calculated in the pose calculation 
module (218) [0053]. The same rationale for combining as applied to claim 1 is 
incorporated herein. 

3. Claims 5-10, 18, and 19 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Ohshima et al. (2003/0032484 A1) in view of Richey (5,310,794), 
Lyons (6,181,343 B1) and Sauer (2002/0140708 A1) as applied to claims 1-4 above, 
and further in view of Santodomingo et al. (7,038,694 B1). 

The modified invention of Ohshima teaches the limitations of claims 5-10, 18, 
and 19 with the Exception of explicitly disclosing wallpaper. However, Santodomingo 
teaches data defining one or more graphic objects are automatically generated in a 
repeatable manner within each of a plurality of visual tiles of a simulated space. 

In regards to claim 5, the method/system of Santodomingo produces an automatic 
scenery generator to produce landscape as a background [col. 7 lines 51-58]. The 
techniques of Santodomingo enable the use of tiles in a more versatile fashion over any 
type of surface, such as a fictitious world, or a simulated wall [col. 8 lines 27-31]. The 
terrain classes designated at the corners of a tile are used to establish a 2D scenery 
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texture for the tile. A texture may be derived from a digital aerial photograph or other 
graphical rendition of a desired surface for the tile. A single scenery texture can be 
used for a tile with all four corners designated with the same terrain class [col. 8 lines 
41-47]. A number of graphically designed texture variations of the same terrain class 
are provided, so that different texture variations can be applied to adjacent tiles that 
have the same terrain class without creating a surface on which repeating patch of 
exactly the same texture is evident [col. 8 lines 52-56]. As illustrated in Fig. 8, 
Santodomingo illustrates wall texture. 

Therefore, it would have been obvious to one of ordinary skill in the art to 
implement the automatic scenery generator of Santodomingo with the modified 
invention of Ohshima in order to provide realism to a virtual environment i.e., the display 
assembly of Richey [Santodomingo: col. 1 lines 15-17]. The scenery texture can be 
implemented by the computer graphics system (6) as shown in Fig. 32 of Richey, and 
displayed on the display walls. 

In regards to claims 6-9, Ohshima teaches integrating real objects within a mixed reality 
system (see rationale of claim 1). Furthermore, it would have been obvious to one of 
ordinary skill in the art to place real objects, such as an operable door and windows with 
shutters, into the display assembly of Richey in order to create realism of the 
augmented reality environment. 
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In regards to claims 10 and 18, the techniques of Santodomingo enables the use of tiles 
in a more versatile fashion over any type of surface, such as a fictitious world, or a 
simulated wall [col. 8 lines 27-31], Furthermore, each tile may instead correspond to 
any desired subdivision [col. 8 lines 10-12]. Therefore, it would have been obvious to 
one of ordinary skill in the art to display fictitious worlds behind the real objects, such as 
a door or window, to provide realism to the user within the display assembly of Richey. 
Furthermore, by having the ability to apply different textures to difference parts of the 
display, the walls (i.e. displays) of Richey would be apply to simulate different fictitious 
worlds, i.e., a plurality of different rooms, of say, within a home. Furthermore, it would 
have been obvious to one of ordinary skill in the art to "decorate" such rooms with the 
real objects as taught by Ohshima, to provide a realism within the augmented reality 
environment. 

In regards to claim 19, the techniques of Santodomingo enable the use of tiles in a more 
versatile fashion over any type of surface, such as a fictitious world, or a simulated wall 
[col. 8 lines 27-31]. Therefore, with the same rationale as applied to claim 5, the 
fictitious world can be an alleyway. 

4. Claims 21, 24, 26, and 28 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Ohshima et al. (2003/0032484 A1) in view of Richey (5,310,794), 
Lyons (6,181,343 B1) and Sauer (2002/0140708 A1) as applied to claim 1 above, and 
further in view of Latypov et al. (6,563,489 B1). 
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The modified invention of Ohshima teaches the claimed limitations of claims 21, 24, 
26, and 28 with the exception of including a computer-controlled sensory generator, 
other than a display. However, Latypov teaches a system for immersing a user in a 
virtual reality that includes controlling the temperature and air content. 

Referring to Fig. 13 of Latypov, the system of Latypov comprises means (30) to 
control temperature and air content in the sphere (claims 26, 28), which is an air 
conditioner (claims 21, 24) [col. 6, lines 29-31]. Additionally, the sphere of Latypov 

Therefore, it would have been obvious to one of ordinary skill in the art to include 
the temperature and air generating conditions of Latypov within the display assembly of 
Ohshima in view of Richey and Lyons because the added environmental changes (i.e. 
additional sensory) enhances the virtual environment to portray to the user that he/she 
is in a real atmosphere. 

5. Claims 21, 24, and 25 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Ohshima et al. (2003/0032484 A1) in view of Richey (5,310,794), 
Lyons (6,181,343 B1) and Sauer (2002/0140708 A1) as applied to claim 1 above, and 
further in view of Tanide et al. (6,201 ,516 B1). 

The modified invention of Ohshima teaches the claimed limitations of claims 21, 
24, and 25 with the exception of including a computer-controlled sensory generator, 
other than a display. However, Tanide teaches a system for sensation. 
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Tanide teaches the operation floor (66) [Fig. 11, 12] is controlled via the motion 
drive device (22d) (claims 21, 24). The floor (66) can be vertically moved, twisted, tilted, 
or displayed (claim 25) [col. 12, line 61 - col. 13, line 21]. 

Therefore, it would have been obvious to one of ordinary skill in the art to include 
the floor operation of Tanide within the display assembly of the modified invention of 
Ohshima because the added environmental changes (i.e. additional sensory) enhances 
the virtual environment to portray to the user that he/she is in a real atmosphere. 

6. Claims 21 and 27 is rejected under 35 U.S.C. 103(a) as being unpatentable over 
Ohshima et al. (2003/0032484 A1) in view of Richey (5,310,794), Lyons (6,181,343 B1) 
and Sauer (2002/0140708 A1) as applied to claim 1 above, and further in view of 
Dowling et al. (2003/0057884 A1). 

The modified invention of Ohshima teaches the claimed limitations of claims 21, 
and 27 with the exception of including a computer-controlled sensory generator, other 
than a display. However, Dowling teaches system consisting of a virtual world in 
coordination with information received from the display. 

Dowling teaches using LED lighting systems (or other types of lighting systems) 
for enhancement of computer game where there is a user in a real world environment 
that surrounds the display screen [0037]. The computer software need not be a game, 
but of any type of computer application [0036]. In use, the illumination system can be 
used to provide information to the user (101) in response to or in coordination with the 
information being provided to the user (101) by the video display (104). One example of 
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how this can be provided is in conjunction with the user playing a computer game on the 
computing device (103). The light system may be used to create one or more light 
effects in response to action on the video display (104) [0049]. 

Therefore, it would have been obvious to one of ordinary skill in the art to include 
the floor operation of Dowling within the modified display assembly of Ohshima because 
the added environmental changes (i.e. additional sensory) enhances the virtual 
environment to portray to the user that he/she is in a real atmosphere. 

Conclusion 

THIS ACTION IS MADE FINAL Applicant is reminded of the extension of time 
policy as set forth in 37 CFR 1.136(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within 
TWO MONTHS of the mailing date of this final action and the advisory action is not 
mailed until after the end of the THREE-MONTH shortened statutory period, then the 
shortened statutory period will expire on the date the advisory action is mailed, and any 
extension fee pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of 
the advisory action. In no event, however, will the statutory period for reply expire later 
than SIX MONTHS from the mailing date of this final action. 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Michelle K. Lay whose telephone number is (571) 272- 
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7661 . The examiner can normally be reached on Monday through Thursday from 
7:30am to 5:00pm. The examiner can also be reached on alternate Fridays. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Kee M. Tung, can be reached at (571) 272-7794. The fax phone number for 
the organization where this application or proceeding is assigned is 571-273-8300. 

Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic . 
Business Center (EBC) at 866-217-9197 (toll-free). 




